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ABSTRACT 



With technology changing rapidly, instruction on its use 
must change rapidly as well . Instructional designers are thus increasingly 
dependent on the use of rapid application development (RAD) procedures for 
producing timely instruction related to technology use. In recent years, the 
use of self-paced instruction in the educational environment has been gaining 
popularity for teaching technology skills. This research examines the 
feasibility of RAD in the design of two undergraduate pre-service teacher 
technology courses. The study was conducted over a three-semester period 
during which a needs assessment was conducted. Eleven units of instruction 
were prototyped, tested, modified, retested, and the course was fully 
implemented. The total number of participants included 570 students, six 
instructional designers/teaching assistants, one lead designer, and four lab 
assistants. Results of the study showed significant differences in usability, 
scores between the second and third semester. Both positive and negative 
experiences with RAD are discussed. (Contains 14 references.) (Author/AEF) 
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Abstract 

This research examines the feasibility of rapid application development (RAD) in the design of two 
undergraduate pre-service teacher technology courses. The study conducted over a three-semester period 
during which a needs assessment was conducted; eleven units of instruction were prototyped, tested, modified, 
retested; and the course was fully implemented. The total number of participants included 570 students, six 
instructional designers/teaching assistants, one lead designer, and four lab assistants. Results of the study showed 
significant differences in usability scores between the second and third semester. Both positive and negative 
experiences with RAD are discussed. 



Introduction 

With technology changing rapidly, instruction on its use must change rapidly as well. Instructional 
designers are thus increasingly dependent on the use of rapid application development (RAD) procedures for 
producing timely instruction related to technology use. In recent years, the use of self-paced instruction in the 
educational environment has been gaining popularity for teaching technology skills. A growing number of self- 
paced instructional formats are now readily available including books, tutorials, and video series, to name a few. The 
benefits of self-paced instruction reach an increasingly larger audience as more people take part in distance learning 
environments, which may in whole or in part employ self-paced features. Despite the benefits, research shows that 
there remains a stigma surrounding teaching a course via a ’distance’ format (Betts, 1998). Further investigation 
shows that much of this reticence comes from the initial, up-ffont time investment and the work involved by the 
person(s) designing and teaching the course (Wollcott & Betts, 1999, p. 35-6). 

In these diverse, fast-paced, and quickly changing educational settings, the use of RAD procedures in 
creating highly usable self-paced instruction is advantageous. How then do we implement RAD for self-paced 
instruction, and how do we make the self-paced instruction effective, efficient, and appealing, and how do we do 
that in short order? This study addresses these overarching questions. 

Review of the Literature 

In the 1960s, the personalized system of instruction (PSI) method of self-paced instruction was developed 
by Fred Keller [Keller & Sherman, (1982)]. PSI was designed and tested to address researchers’ concerns that 
teaching was primarily focused on the teachers both actively, and passively presenting information to a passive, and 
at times somnolent, student. According to Sherman (1992), this type of teaching tends to neglect what and how the 
student learns. It also assumes that the student, as a passive learner, will one way or another receive and understand 
the information simply because the information has been presented. Teaching styles during that time also 
encouraged negative reinforcement as behavior/leaming modification. Sherman goes on to state that, “if a high 
frequency of behavior is to be encouraged so that progress can be selectively rewarded, punishing errors is the 
wrong way to go about it” (p.59). Both PSI and self-paced instruction address these concerns. 

According to Kemp, Morrison, & Ross (1998), PSI is also often referred to as “the Keller Plan” and is often 
used when planning and implementing a whole course of instruction. At the time of its inception, this method was 
based on the use of a textbook, with assigned readings, followed by questions and problem solving. With the advent 
of microcomputers and especially when these are connected to the Internet, the Keller Plan can be more creatively 
applied — often with dramatically positive results. 

Self-paced instruction has long been used to teach computer applications such as word processing, 
spreadsheets, MS Power Point, etc. Technology, however, is changing so rapidly that it often requires frequent 
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updates to training materials. Unfortunately, self-paced materials are time consuming to produce and thus present a 
developmental challenge in a dynamic technological world. Even though the designing of self-paced instruction 
brings with it several unique challenges, there are several advantages to self-paced instruction that make it worth of 
the time expended in development (Hannifin & Peck, 1987). Most importantly, self-paced formats allow individuals 
to progress though the material at their own rate. In the classroom environment frees the teacher to help those 
students who need augmented and/or more ancillary instruction. In both the classroom, and the distance-learning 
environments, self-paced instruction allows the learners to access the material at times that fit their individual 
schedules. 

It is common practice for software development environments to follow RAD models in creating their 
software products (Galitz, 1997). Although research has been conducted on self-paced instruction, there is little 
specific research on the development of self-paced instruction using RAD procedures in academic environments. 

Rapid application development is a term used to describe production of software and training products in a 
series of development cycles that move quickly from initial prototypes to a fully developed product. In these 
scenarios a prototype of the software is quickly created, tested for usability, and then revised. This cycle is repeated 
until the final product has been developed. Reigeluth & Nelson (1997) describe a RAD approach in an emerging 
paradigm of instructional systems design (ISD) they call ASEC (Analysis, Synthesis, Evaluation, and Change). 
Design and development phases move through an “iterative series of ASEC cycles for progressive sets of 
instructional decisions" (Reigeluth & Nelson, 1997, p. 31). 

The ASEC process allows a designer/developer to focus on how a learner or user responds to instruction as 
it is being developed. The Evaluation Step in the ASEC process allows for this type of leamer/user feedback, 
(important to learning more about the usability or learner-friendliness of the product). "User-fnendly is a term that 
describes how technically easy a product is. "Learner-friendly" describes how easy an instructional product is for a 
learner to use (Lohr, 2000). The International Standards Organization (ISO) defines usability with the descriptors 
effectiveness, efficiency, and satisfaction. Lohr (2000) defines usability and learner-friendliness with effectiveness, 
efficiency, and appeal, based on similar definitions by other design researchers ( Flagg, 1990, Nielson, 1993, 
Shackel, 1991; Tessmer, 1993) . 

Research Questions 

This study focuses on the development of self-paced pre-service teacher technology training using a RAD 
approach. Research questions include: 

1 . How usable was the instruction from student, instructional designer/Teaching Assistant (ID/TA), lead designer, 
and lab assistant perspectives? 

2. Did usability perceptions change during the RAD process? 

3. What are the advantages and disadvantages of using a RAD process in an academic setting? 

Methodology 

Participants 

This research was conducted at a medium-sized Western university and involved 570 pre-service teachers 
enrolled in two levels of technology- integration courses. Other participants in the study included six instructional 
designers/teaching assistants (IDs/TAs), a lead instructional designer, and four lab assistants. Out of 657 total 
students enrolled in both 200-level, and 300-level levels during the two semesters, a total of 570 students 
participated in the study. Of the 570 students who filled out the surveys, 316 students were from the 200-level 
classes and 254 students from 300-IeveI classes (described later in this paper). At the end of each semester, four 
students from each class were randomly selected to be interviewed (a total of 1 1 8). 

Course Description 

Two one-hour pre-service teacher technology courses were developed for this study, referred to as the 200- 
level and 300-level courses. The 200-level course was an introduction to technology for freshmen or sophomores. 
The 300-level course was a more advanced course for juniors and seniors. In both courses combined, there were a 
total of 1 1 units of instructions to be created. The topics included, email, uses of the World Wide Web (WWW), and 
word processing. 

All sections were taught in three formats: workshops, open-labs, and via self-paced instruction. The three 
formats broke down as follows: The workshops consisted of instruction, both verbal and visual, where each project 
was completed in front of the students. In this format many of the students could follow along step by step. The open 
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labs were set up so that any student could get individual instruction. For the aspects of each project where a number 
of students were struggling over the same component, this was noted so that it could be addressed in a redesign 
between semesters. The lab assistants were also present during open labs to help students one-on-one with their 
individual difficulties. The self-paced instruction was taught via the web sites set up for each of the two courses. 
Every student was required to have an e-mail account and access to the Internet. The students could access the 200- 
level and 300-level web sites where all the 1 1 units of instruction were hosted. To view these units of instruction, 
visit these Web sites; www.edtech.unco.edu/et34x/ and www.edtech.unco.edu/et24x/. 

Procedures 

The study was conducted over three semesters (Spring of 1999, Fall of 1999 and Fpring of 2000). The 
topic covered as well as the formats of instruction were gleaned from a pre-service teacher needs assessment 
conducted in the spring semester of 1999. Three students (pre-service teachers), three graduate -level teaching 
assistants, and one faculty member were interviewed about the effectiveness, efficiency, and appeal of the 
undergraduate pre-service teacher technology courses at that time. These interviews revealed the following: a) there 
existed a lack of instructional consistency in courses (different instructors teaching different topic in different ways); 
b) the instructional materials were outdated; c) students came to the class with different proficiency levels ranging 
from computer illiterate to highly skilled computer users, but all students were expected to learn at the same pace; d) 
students were required to attend all classes even if they knew, and were proficient with, the software being taught; e) 
many students had scheduling conflicts that could have been prevented with the implementation of a self-paced 
format. 

The needs assessment team identified self-paced instruction as a possible solution to these issues, since SPI 
not only allows for learners to progress at their own pace, but allows for greater standardization of the instructional 
content as well. 

Beginning with the fall semester of 1999, a team of three teaching assistants ( TA’s) and a lead 
instructional designer (ID) planned the development and implementation of RAD to the instructional material 
needed for the courses. The team met weekly intervals to plan and review their work, discuss problems, concerns, 
and progress. The team also kept a record of their observations. The actual RAD course development was 
performed throughout the first semester, but a revising process (implementing feedback from the first semesters' 
participants) continued between the semesters as well. Each of the teaching assistants developed approximately 
one-third of the total units of instruction. 

Every week each of the developed units of instruction would be presented to the team members who would 
^collectively review what each TA had designed, test the instruction, and make suggestions. At this time the lead ID 
would also make suggestions. After this open-forum-type of critiquing was finished, the TA in charge of each unit 
of instruction would make the necessary revisions, test it, and then re-present it to the group the following week 
where the rest of the team members would again review the TA's work, usability test the unit of instruction, and give 
their final suggestions. The TA would then make any necessary final revisions, transfer the finalized unit of 
instruction to the course web site for those who were doing the course via self-paced instruction, and the instructor 
would also present it to the class as an in-class instruction module. This process of designing, developing, open- 
forum critiquing, revising, and usability testing was followed for each of the 1 1 units of instructions. Thus, every 
two weeks and within a two-week span, the RAD model was used with two individual units of instruction. 

Instruments 

Quantitative and qualitative data were collected from the following sources: students participating in the 
course; teaching assistants, all of whom not only taught the course, but also monitored the lab sessions, and some of 
whom (instructional designers) also helped design the course; the lead instructional designer who managed the 
design and development process, and the lab assistants who interacted with the students while the students 
participated in the instruction. Both the lead instructional designer and the IDs/TAs kept journals for a 12-week 
period. Lab assitants were informally interviewed. Qualitative student data was obtained using 16 open-ended 
questions included in an end-of-the-semester questionnaire, from 120 student observations, and from 118 student 
interviews. 

All qualitative data were transcribed, and thoroughly read and re-read. Different colored markers were used 
to highlight words and statements which fell into the three main research categories: effectiveness, efficiency, and 
appeal. An overall theme and sub- were then identified for each category. 

Quantitative data were collected using 43 Likcrt-type items included in an end-of-the-semester 
questionnaire. The five-point Likert scale ranged from “strongly disagree” to “strongly agree.” The items were 
created for each of the three main domain areas: effectiveness, efficiency, and appeal. Factor analysis was used to 
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verify the structure of the above mentioned survey questionnaire. A structure matrix of a three-factor oblique 
rotation (Kappa = 4) of a principle-components solution was found. The factors were learning and educational value 
(effectiveness), resources (efficiency), and self-paced instruction (appeal). Problematic items i.e., loaded on two 
factors or demonstrated a factor loading under .30 on any factor, were not used as part of the composite scores. Item 
analysis within each factor demonstrated strong reliability, and hence stability of the survey as a whole. Reliability 
coefficients for the 200-level courses were found to be OCenectivc = 0.9301, cXefr,cicncy = 0.8071 and CXappcai = 0.8070 
respectively. Reliability coefficients for the 300-level courses were found to be Oencctive= 0.9121, cXefTiciency= 0.7270 
and (Xappcai = 0.8775 respectively. The final composition of the domains suggest a degree of validity. 



Results 

Students perceptions of usability 
Effectiveness 

Qualitative analysis of the randomly selected interviews and open-ended questions found a strong majority 
of positive responses from students related to the effectiveness of the course. Students mentioned liking the course 
for a variety of reasons that were categorized as educational and learning value. Data were coded into the following 
sub-themes: content learned was applicable to the classroom; knowledge was gained; portfolios were developed; 
topics could be freely chosen; projects were relevant; the overall format was convenient; he experience was isolating 
and often not challenging enough. Most of the positive comments from both levels were associated with how precise 
the instruction was and how useful it would be in a classroom. Most of the negative comments were associated with 
dislike for specific topics that were considered repetitive (ASSURE), too difficult (Photoshop), or too easy (email). 
The most common statements used by students were as follows; “I learned a lot in this class.”; “I gained more 
knowledge, new ideas, and ways to use technology.”; and, “1 feel comfortable with technology.” All of the students 
randomly selected to perform a task representing the semester’s work performed satisfactorily. All of these same 
students felt the textbooks were a ’’waste of time". 

Quantitative data for effectiveness was calculated by averaging the percentage of agree and strongly agree 
scores for fall and spring semester for each course level on specific questions. Quantitative results were similar to 
qualitative results with a strong majority of students (86% of fall and spring 200-level and 72% of the fall and spring 
300-level) agreeing or strongly agreeing that the course was educationally valuable. Eighty nine percent of the 200- 
level and 72% of the 300- level agreed or strongly agreed that skills learned were important. Fewer students 
indicated they learned a lot (61% of the 200-level, 69% of the 300-level). As indicated by the qualitative data as 
well, both levels felt that the instruction, given both in-class and on-line, was precise and clear. These students 
indicated that they would use the technological skills learned during this course both in their future classrooms for 
instructional delivery, and for their own personal classroom management use and gains. They stated that this usage 
would include programs like PowerPoint (200-level) and Web design (300-level, also suggested as a way to gain 
employment) and incorporate the design and graphics knowledge they had learned. These students indicated that the 
course helped them not only learn skills with which they were previously unfamiliar, but also to learn how to 
integrate technology into their classrooms. At the end of the course, a majority of students felt very comfortable with 
technology 

Efficiency 

In the Qualitative analysis the Value of resources in terms of access and ease of use emerged as the key 
theme or descriptor for questions related to efficiency. Data were coded, then grouped into these sub-themes: online 
materials, open labs, workshops, instructors and lab-consultants. Two-hundred-level and 300-level students were 
most positive about having a choice of learning format (workshop, open-lab, or self-paced), and most negative about 
specific projects and isolation. The 300-level group felt strongly that they had too much work to do for a one-credit 
course. Qf those who were interviewed that did not like the course, two main reasons were given. Some mentioned 
they did not like the self-paced format because of isolation and problems with self-discipline. Others mentioned they 
didn't believe the technology would be available to them in the classroom. All students felt the textbooks were not 
helpful. Half of the students in the first semester wanted more project examples when interviewed at the end of the 
fall semester. Surprisingly just a little under a half of the students interviewed in the spring still indicated the need 
for more examples (despite the development of examples and non-examples for each unit of instruction that took 
place during the spring semester). 
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Typical comments during student interviews included: “The on-line materials were precise and easy to 
understand because of the step by step instructions”; “The rubrics/or grading criteria were too easy”; “I would like to 
see more examples of finished projects” ;“PhotoShop was too confusing”; “In open-labs I could get personal 
attention and help to complete my projects”;** Workshops were very helpful for difficult projects like PhotoShop, 
Web design and PowerPoint”; “The instructors were very approachable, accessible and always available to help 
me”; “The instructors were knowledgeable.”;‘‘The lab consultants were there to help me when I got stuck.” 

Quantitative data for efficiency was calculated by averaging the percentage of agree and strongly agree 
scores for fall and spring semester for each course level on specific questions. Quantitative data was similar to 
qualitative with a higher percentage of neutral responses and a lower percentage of responses in the agree and 
strongly agree categories. Approximately 60% of the 200-level responses were positive about both the instructors 
and the self-paced instruction. Approximately 75% of the 300-level responses were positive about the instructors 
and the self-paced instruction. Open labs and workshops received lower scores, with less than half of the 200-level 
students positive about either format. The data was similar for the 300-level who had a slightly higher percentage of 
agree or strongly agree responses (59% for workshops and 55% for open-labs.) Of particular interest is that only 
58% of the 200-level and 59% of the 300-level felt they had enough self-discipline for the self-paced format. 

Appeal 

Of the three usability variables, appeal had the most positive student responses. Quantitative data related to 
appeal was more positive than the qualitative data regarding overall course appeal. From the qualitative data, a 
a strong majority of students interviewed mentioned liking the self-paced nature of the course. Not surprisingly, the 
key theme that emerged from the questions was a positive regard for the self-paced format. Self-paced thus emerged 
as a key descriptor for appeal based on data that was coded then grouped into following sub-themes: freedom, self- 
discipline, convenience, isolation, and not challenging. Both levels of students mentioned the self-paced format gave 
them the freedom and flexibility to go at their own rate and, hence, made them feel less pressured and felt they had a 
lower stress level than if they had taken the course via a traditional in-class format. The self-paced format was 
particularly helpful for the students who already knew most of the tools. The self-paced format also required 
students to apply better time-management skills. Again, some students did not like the self-paced because they felt 
the format lacked student/teacher interaction as well as class-related communication with fellow classmates. Typical 
comments included: **I liked the option to attend workshops open labs or follow self-paced for projects that I was 
comfortable with”; “It was nice to know that help was available whenever required”; “I learned to manage time 
effectively”; “I sometimes tended to procrastinate my work”; “I liked the convenience of working at my own time 
and pace”; *‘I liked the fact that I did not have to come to the class if I knew the project”; “The fact that I did not 
meet officially made me feel isolated”; *‘Projects like word, e-mail, WWW were too simple and basic”; “I knew 
most of the software and hence was hard to stay motivated.” 

Quantitative data for appeal was calculated by averaging the percentage of agree and strongly agree scores 
for fall and spring semester for each course level for specific questions. Results related to appeal were quite high for 
both the 200 and 300 level students. Approximately 90 % of the students agreed or strongly agreed to statements 
regarding the overall usefulness of self-paced instruction. For example 97% of 200-level students and 89% of 300 
level students indicated liking self-paced instruction. Ninety five percent of the 200-level students and 89% of the 
300-level students indicated the self-paced instruction was helpful. 

Teaching Assistants/Instructional Designers 

The instructional designer/teaching assistants were asked to keep a journal during the fall semester to 
record two types of information: 1 ) student behaviors and attitudes, and 2) issues relating to the format and accuracy 
of instruction. Perceived effectiveness, efficiency, and appeal data were categorized and grouped. Observations 
related to the effectiveness of the course included noticing that students were often copying the assignment example 
instead of generating their own solutions, and were not checking the Web site bulletin board. Observations related to 
the efficiency included the lack of student attendance at workshops. For example, students would elect the self- 
paced format, wait until the last minute, and then discover they could not do the assignment without help. 
Observations related to appeal included reports that many students were unhappy with scheduling issues. These 
students said the classes did not meet during convenient times. 

Several actions were taken immediately prior to the next semester to address these issues: 1) the free-for-all 
schedule was eliminated requiring students to sign up for a class at a specified time; 2) students were required to 
attend workshops unless they had turned in the self-paced instruction; and 3) class listserves were used as a 
communication vehicle in addition to the message board. During the following semester these problems were not 
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mentioned by the students. One issue however that remained a problem was the need for the Web site to provide 
more and varied project examples since students continued to turn in assignments very similar to the examples used. 

Lab Assistants 

The lab assistant interviews indicated mostly problems with efficiency and effectiveness . Lab assistants 
described themselves as being bombarded the day an assignment was due. These students felt that they had to teach 
students how to do their projects. The interviews revealed that over half of the lab assistants did not know about the 
self-paced materials or where they could be located. Those who did know indicated that most of the students did not 
have the web-site address easily accessible. Lab assistants addressed this problem by writing the class website on lab 
whiteboard. The one problem identified in the Spring semester interviews with lab assistants was the difficulty 
students were still experiencing with the Web site URL. Therefore in the third semester of the project, the web site 
was given a new and simpler URL, and desktop Web site icons were created to allow students to access the site by 
clicking the icon instead of typing in a lengthy url (http://www.edtech.unco.edu/et34x/def34x.htmn . 

The Lead Instructional Designer 

The Analysis, Synthesis, Evaluation, and Change interpretation of RAD was documented and evaluated on 
a weekly basis by the lead designer to determine overall effectiveness of the RAD approach. Time spent, 
management issues, technical issues, and student behaviors were noted for all eleven cycles of development during 
the fall semester. Of each of the steps, the lead designer was most disappointed in the inability to implement learner 
testing in the Evaluation step. Lack of time prevented the IDs to involve students in the development of the 
instruction. Therefore the learner-testing phase was actually a design-team testing phase. The process however 
seemed adequately effective even without student involvement in the two-week testing phase. The units of 
instruction were delivered on time with few errors. 

The efficiency of the RAD process was assessed in part by recording time-spent on specific activites. The 
lead designer was not prepared for the amount of time it took to set up the course templates and create master 
schedules for open labs and workshops to accommodate student schedules. Three full days were spent working on 
the schedule alone. These cycles included a brief analysis step (identification of a specific task and content needed), 
development of instruction (seven hours), testing by the team (1-2 hours), revision and Web posting by the 
designer (8 hours), team re-testing and revision, changes (3 hours), and implementation. The lead designer also 
regretted her decision to use Microsoft Word as the Web-site authoring tool because it was less stable on the server 
than were other Web authoring programs. Microsoft Word was chosen because most IDs/TAs were not proficient 
with Web authoring and needed a very simple tool. Unfortunately this backfired with numerous server problems that 
took up an additional four hours per week to address. 

The more interesting aspect of the appeal data relate to the RAD process came from the lead designer's 
observations of ID’s/TA’s gradual acceptance of both the format of instruction and the RAD process. Early in the 
first semester students were highly frustrated with the process of rapid development. They were unfamiliar with the 
precision self-paced instruction required as well as the demand for user-friendly writing. Combined with low 
workshop and open lab attendance during the first few weeks of the semester, these TA/IDs were openly frustrated. 
By the middle of the semester they voiced extreme dissatisfaction with the process and structure. During the second 
half of the semester they gradually gained more acceptance of the format and showed interest in making 
improvements. Also noted was their improved ability to write learner-friendly instruction. 

Change In Usability Between Semesters 

Lead designer, T As/IDs, and lab assistants all noticed improvement in the course during the Spring 
semester. These observations are in part supported by multivariate analysis of variance (MANOVA) results. A 
MANOVA was used to investigate differences between the fall 1999 and spring 2000 semesters with respect to the 
factors: effectiveness, efficiency and appeal. Significant MANOVA results were followed by discriminate analysis. 
A significant difference was found in the ET 200-level data between the fall of 1999 and the spring of 2000 
semesters (F (3,301) = 8.356, p<0.00, Wilks’ lambda=0.923). A stepwise discriminate analysis was used to 
determine if there was a specific variable contributing the strongest influence to this difference. It was found that 
only the resources factor — efficiency - (e.g., workshops, open labs, and self-paced instruction) that contributed 
most to a change between semesters (F (1,303) = 22.1 1, p<0.00). 

In comparing the fall and spring ET 300-level courses, significant differences in all three factors (F (3,240) 
= 19.702, p < 0.00, Wilks’ lambda = 0.802) were found. A stepwise discriminate analysis was used to determine if 
there was a specific factor contributing the strongest influence to this difference. This analysis indicated that all the 
three dependent variables were equally responsible for the differences between fall of 1999 and the spring of 2000 
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semesters. The stepwise discriminate analysis showed that Appeal made the biggest difference among semesters 
(F=45.568, Wilks’ Lambda=.955). Appeal was followed by Effectiveness (F=41.426, Wilks’ Lamda = .941). 
Efficiency made the smallest difference among semesters (F = 8.18, Wilks’ Samda = .831). 

Discussion 

That significant differences were found in usability perceptions between the second and third semesters 
might suggest that RAD played a role in improving instruction. Changes to instruction were made based on data 
gathered during the semester. For example clarifications in assignments, additional examples, the correction of 
errors, all improved the quality of the experience. Many of the concerns and complaints expressed by TAs and 
students alike during the first semester were not voiced during the second semester. 

There are however, many possible reasons for the significant differences that took place between the 
second and third semester that may not be attributable to RAD. A more positive or motivated group of students may 
have been enrolled in the second semester. TAs may have conveyed more confidence, thereby influencing the 
attitudes of the students for a variety of reasons. Most obvious would be that the materials were already developed 
and error-free, therefore TAs had more time to devote to individual students, and were also more confident that 
fewer mistakes would cause student confusion. The simple change in required scheduling had perhaps the biggest 
impact as many students were dissatisfied with the open schedule format of the first semester. This schedule change 
was not a function of RAD. 

The key benefits of RAD in this study appears to be improved instruction that lasts for a longer period of 
time. There has been much cited in the research about the low return on investment of self-paced instruction 
(Wolcott & Betts, 1999). Initially, in this study, that was the case as well; however, because the course content has 
remained fairly stable over the past year and a half, the time expenditure has been amortized very effectively. As this 
article is being written (halfway through the third semester after the RAD took place) the researchers have found 
that the time expended for evaluation and revision have been substantially less than if the courses were being taught 
via a more traditional format. For the spring 2000 semester, approximately a week was spend in completing a 
cursory analysis of the data from the previous semester’s surveys; in reevaluating the course philosophies and 
content in light of the surveys; and in revising the Websites, syllabi, rubrics, and schedules. In the summer 
semester, there were no changes made other than changing the dates and times on both the syllabi and the course 
schedule, which took approximately two hours for both courses. In the fall 2000 semester, a total of approximately 
two days was spent in the review, analysis, and revision process. All told, for two different courses that serve so 
many students, this time expenditure is quite modest. 

The key drawback of RAD in this study was the need to act quickly. In this study hasty decisions had long- 
lasting ramifications. When the study began, the lead ID drew up the original course outline and even created the 
initial Website shell for the courses using a very simple authoring tool, Microsoft Word. This was done because the 
TA/IDs had little experience with developing Web based instruction and would need a familiar and easy 
development tool. Because the Lead ID did not want application specific ramp-up time to impede the IDs progress, 
it was decided that templates could be created and used. Because Microsoft Word is a fairly basic web-authoring 
tool the software became a limiting factor once the demands of the site require more complexity. The team 
recommends beginning with a more robust web-authoring software capable of accommodating a broader range of 
Web site maintenance tasks, such as frequent revisions to instruction and use of different browsers and Web editors. 

Also of interest to future RAD projects in academic environments is the effect the approach has on novice 
instructional designers. Because rapid application development is just that, rapid, it brings with it a set of dynamics 
with which most academic settings are not usually confronted. In this study, the lead ID had been through this type 
of development numerous times and, as a result, was mentally prepared for the frustration that is typically a part of 
the designing, developing, open-forum critiquing, revising, and usability testing process. Individual and team 
fhistration is so inherent in the RAD model that it can almost be mapped out and defined as part of the cycle. In this 
study’s cycle, the TA/IDs became quite frustrated with the intensity and seeming lack of progress early on. However, 
as the process continued and the fresh, never-tried-before units of instruction were introduced to the students they 
seamlessly learned the material and skills and produced the hoped-for and required results. The TA/IDs then began 
to more clearly understand how RAD works, and as a result, became much more enthusiastic about its application 
and results. 

Conclusion 

Overall, quantitative and qualitative data collected from students, TAs/IDs, the lead ID, and lab assistants 
suggest that the course was learner-friendly. The data identified areas for improvement as well as areas where 
students were highly satisfied with the quality of instruction in both the 200-level and 300-level courses. The 
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importance of providing instruction that is neither too easy or too difficult was identified, as both were mentioned as 
negative aspects of the course. The need to help students think creatively and find ways to use the tool for their 
unique instructional purposes must be improved, perhaps through many and varied examples and non-examples. 
Scheduling problems and difficult Web site access negatively influenced student’s perceptions of efficiency and 
appeal. The self-paced format, however, was considered highly appealing. Though many students considered 
themselves self-disciplined, the data suggested the need to implement a structure for students that imposed due 
dates, designated class times, and the requirement to have self-paced instruction completed prior to a workshop. 

Although RAD has found an almost hallowed place in fast-paced corporate settings, we were reticent about 
its usefulness in an academic setting. The results of this study, however, have shown that RAD can be applied quite 
effectively. The results have shown that in the right application, time-dependant course material can be delivered 
efficiently, effectively, and in an appealing manner using RAD. The caveat is that all aspects of the process should 
be well thought out, critiqued, revised, and usability tested. 

Several areas of future research related to this topic should be explored. For one, the importance of 
instructional graphics needs to be examined. Because instructional graphics are so time-consuming to develop, an 
important consideration is the optimal amount of graphics to include as well as the type of graphics for RAD of self- 
paced instruction. For example, should graphics show an entire computer screen, or should they display just a button 
or pull down menu? Additionally, what are the most learner-friendly writing conventions? Is ’’select Save As from 
the File pull down menu" as effective as File > Save As? Another more challenging question asks how pre-service 
teachers can be taught to think creatively when applying technology to their unique instructional management or 
development goals. The excuse of not needing to learn about technology because one already knows how to use a 
particular tool isn't valid when the focus is on learning to use the tool to enhance instruction. These questions and 
many more will provide more insight into learner-friendly development of pre-service teacher technology courses. 
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